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Organizational  Structure  and  Dynamic  Information  Awareness 

In  Command  Teams 

Abstract 

The  Navy  has  embraced  the  concept  of  network  centric  warfare  as  a  means  to  develop 
innovative  and  effective  command  and  control  (C2)  structures  for  the  future.  One  such  C2 
structure  is  FORCEnet.  Modeling  of  various  FORCEnet-derived  structures  produced  a 
prediction  that  a  C2  structure  that  includes  an  intelligence,  surveillance,  and 
reconnaissance  (ISR)  coordinator  would  significantly  improve  mission  performance. 
Network  centric  warfare,  however,  has  increased  the  information  load  commanders  must 
deal  with.  As  part  of  this  effort  we  investigated  the  effects  of  information  load  on  certain 
decision  making  heuristics.  Counter  to  modeling  predictions,  a  FORCEnet  derived 
organization  with  an  ISR  coordinator  was  not  superior  in  performance  to  an  organization 
without  an  ISR  coordinator.  There  was  evidence,  however,  that  increased  familiarity  and 
practice  with  a  structure  including  an  ISR  coordinator  may  produce  findings  supportive 
of  model  predictions.  Findings  also  indicate  that  high  information  load  may  exacerbate 
the  negative  effects  of  certain  decision  making  heuristics. 

Introduction 

As  it  seeks  to  develop  innovative  and  effective  command  structures  for  the  21st  century 
the  Navy  has  embraced  the  concept  of  network-centric  warfare  as  espoused  by  Admiral 
Cebrowski  and  others.  One  such  innovative  command  and  control  (C2)  structure  to 
receive  attention  is  FORCEnet.  The  principles  of  FORCEnet  C2  have  been  described  as: 
“(a)  Distribution  of  forces  to  achieve  tactical  stability;  (b)  Networking  technology  as  an 
enabler  of  new,  evolutionary  organizational  structures,  allowing  distributed  forces  to 
collaborate  at  a  distance;  (c)  Evolution  of  the  tactical  organization  as  it  adapts  to 
changing  demands  of  the  mission;  and  (d)  Collaboration  as  a  way  of  life  . . .,  coming 
from  changes  in  the  overlap  of  responsibilities  in  the  organization  structure  during 
execution  and  increased  collaboration  during  planning”  (Serfaty  et  al.,  2002,  p  2). 

These  principles  guided  a  modeling  and  simulation  (M&S)  effort  conducted  for  the 
Strategic  Studies  Group  XXI  to  investigate  the  effects  a  FORCEnet  structure  might  have 
for  a  large  naval  organization  like  a  battle  group  (Serfaty  et  al.,  2002).  The  M&S  efforts 
predicted  an  increase  in  shared  knowledge  in  a  FORCEnet  structure  which  in  turn  allows 
the  organization  to  respond  more  effectively  to  new  and  unexpected  situations,  making  a 
FORCEnet  structure  more  adaptable.  Overall,  the  modeling  effort  predicts  that  a 
FORCEnet  structure  will  handle  complex  tasks  that  require  coordination  more  effectively 
than  current  organizational  structures.  In  FORCEnet,  collaboration  is  envisioned  as  a 
way  of  life,  and  this  collaboration  provides  an  advantage  in  both  speed  of  command  and 
in  the  adaptability  of  the  organization  to  new  situations. 

Modeling  aspects  of  a  FORCEnet  structure  also  revealed  that  some  structural  changes 
promoted  by  FORCEnet  appear  to  have  a  facilitating  effect  on  mission  performance.  One 
such  structural  change  is  the  presence  of  an  intelligence,  surveillance,  and  reconnaissance 
(ISR)  coordinator,  a  new  command  position  that  would  coordinate  all  theater  sensors  and 


maintain  situational  awareness  for  the  organization.  Modeling  results  indicated  a  25% 
increase  in  mission  performance  when  an  ISR  coordinator  is  present  compared  to  when  no 
ISR  coordinator  was  present  (Serfaty  et  al.,  2002). 

A  primary  goal  of  this  research  was  to  empirically  test  this  model  prediction.  We 
experimentally  contrasted  an  organizational  structure  with  an  ISR  coordinator  to  a 
traditional  Composite  Warfare  Commander  (CWC)  like  structure  where  no  such 
coordinator  is  present,  using  a  man-in-the-loop  simulation.  We  predicted  that  an 
organizational  structure  that  includes  an  ISR  coordinator  to  direct  sensor  assets  and 
maintain  situational  awareness  would  outperform  an  organization  structure  without  such 
a  coordinator. 

At  the  same  time,  the  move  toward  network-centric  warfare  and  the  introduction  of  new 
technologies  have  also  increased  the  volume  of  available  information  commanders  must 
deal  with.  Commanders  face  increasingly  larger  data  sets  that  they  are  expected  to 
integrate  and  interpret  within  increasingly  shorter  periods  of  time.  Research  literature 
(see,  for  example,  Entin,  Kerrigan,  Serfaty,  Klein,  and  Wolf,  1998)  suggests  that  high 
information  loads  can  derogate  situation  assessments  and  decrease  mission  performance. 
To  empirically  investigate  the  effects  of  increased  information  load  on  decision  making 
and  mission  performance  brought  about  by  the  technologies  enabling  network-centric 
warfare  we  manipulated  information  load  across  the  two  organizational  structures.  To 
further  focus  on  the  critical  aspects  of  decision  making  we  created  a  tactical  judgment 
task  that  ran  parallel  to  the  simulation  scenario  and  we  embedded  confirming  or 
disconfirming  information  necessary  for  the  judgment  task  in  the  information  flow.  The 
tactical  judgment  task  was  derived  from  a  task  used  by  Entin  and  Serfaty  (1997). 

Entin  and  Serfaty  (1997)  addressed  the  process  of  sequential  revision  of  beliefs  or 
judgments  in  complex  situations.  They  note  that  C2  structures  provide  decision  makers 
with  opportunities  to  revise  their  tactical  judgments  as  streams  of  information  flow  in  for 
their  consideration.  A  contrast- inertia  model,  based  on  Hogath  and  Einhorn’s  (1992) 
belief-adjustment  heuristics  model,  was  postulated  by  Entin  &  Serfaty  (1997)  to  describe 
participants’  sequential  revision  of  beliefs  when  attempting  to  integrate  pieces  of 
confirming  and  disconfirming  evidence.  Results  indicated  that  the  sequential  order  of 
confirming  or  disconfirming  evidence  had  a  profound  effect  on  participants’  judgments. 
That  is,  participants  arrived  at  completely  different  decisions  depending  on  whether 
confirmatory  evidence  precedes  disconfirmatory  evidence  or  whether  disconfirmatory 
evidence  precedes  confirmatory  evidence.  These  results,  predicted  by  the  contrast-inertia 
model,  are  referred  to  as  the  order  effect. 

We  expected,  based  on  Entin,  et  al.  (1998)  for  example,  that  high  information  load  would 
lower  mission  performance.  We  also  predicted  that  participants  would  revise  there 
judgments  in  accordance  with  the  Hogath  and  Einhom’s  (1992)  belief-adjustment  model 
and  Entin  and  Serfaty  (1997)  contrast-inertia  model.  Although,  past  work  holds  no 
predictions  on  how  information  load  might  interact  with  judgment  revision  or  the  order 
effect  we  offer  two  speculations.  High  information  load  may  obscure  the  confirming  and 
disconfirming  evidence  making  it  difficult  for  decision  makers  to  revise  their  beliefs.  Or 


high  information  load  may  heighten  decision  makers’  stress  levels  leading  to  greater 
susceptibility  to  heuristic  errors  like  the  order  effect.  A  phenomenon  that  can  lead  a 
commander  to  arrive  at  opposite  beliefs  by  the  mere  presentation  of  the  same  evidence  in 
different  orders  deserves  careful  investigation  (Entin  &  Serfaty,  1997). 

Method 


Participants 

Thirty-two  officers  provided  by  the  Surface  Warfare  Office  School,  Naval  War  College, 
Newport,  RI  were  organized  into  eight  teams  of  four  individuals  each.  Most  of  the 
officers  were  male  and  at  the  03  level.  Each  team  spent  about  75  minutes  in  training  and 
75  minutes  in  data  collection. 

Experimental  Design  and  Independent  Variables 

The  experimental  design  manipulated  three  independent  variables.  Organizational 
structure  and  confirmation  order  were  manipulated  as  between-subjects  factors,  and 
information  load  as  a  within-subjects  factor. 


The  between-subjects  variable  organization  structure  was  comprised  of  two  conditions: 
ISR  coordinator  present  and  ISR  coordinator  not  present.  When  the  ISR  coordinator  was 
present  that  position  owned  all  the  ISR  assets  (mainly  unmanned  surveillance  vehicles) 
and  fulfilled  the  job  of  coordinating  the  sensor  picture  and  maintaining  situational 
awareness.  When  the  ISR  coordinator  was  not  present  the  team  member  in  that  position 
was  the  Surface  Warfare  Commander  and  ISR  assets  were  distributed  among  all  team 
positions  and  all  team  members  were  required  to  coordinate  to  manage  the  sensor  picture 
and  maintain  situational  awareness. 


Information  load  was  operationalized  in  a  manner  similar  to  that  described  by  Entin, 
Entin,  &  Hess  (2000)  and  implemented  with  two  levels:  low  and  high.  The  low 
information  load  condition  was  characterized  by  approximately  3.5  messages  per  minute, 
whereas  in  the  high  information  load  condition  team  members  experienced  a  message 
rate  that  was  2.5  times  higher  or  approximately  9  messages  per  minute.  These 
information  flow  rates  were  derived  from  team  communication  rates  observed  in  Entin 
(1999)  and  Entin  et  al.  (2003).  All  information  was  conveyed  to  participants  via 
electronic  mail  messages. 


Confirmation  order  was  manipulated  as  a  between-subjects  independent  variable  and  drew 
on  the  methodology  described  by  Entin  &  Serfaty  (1997).  An  anchor  or  initial  probability 
that  the  enemy  will  launch  a  counter-attack  against  Blue’s  follow  on  forces  was  established 
by  a  written  “intelligence  message.”  The  intelligence  message  indicated  that  the  likelihood 
of  an  attack  was  75%.  During  the  scenarios  the  half  of  the  teams  that  were  assigned 
randomly  to  the  “confirm  -  disconfirm”  order  condition  received  information  via 
Email/Intel  messages  imbedded  in  the  general  Email/Intel  traffic  that  first  confirmed  the 
attack  (three  messages)  and  then  later  disconfirmed  the  attack  (three  messages). 


Conversely,  the  half  of  the  teams  that  were  randomly  assigned  to  the  “disconfirm  - 
confirm”  order  condition  received  Email/Intel  messages  imbedded  in  the  general 
Email/Intel  traffic  that  first  disconfirming  the  attack  (three  messages)  and  then  later 
confirming  the  attack  (three  messages).  The  first  three  messages  were  scheduled  to  appear 
approximately  3,  9,  and  15  minute  into  the  scenario,  and  the  second  set  of  three  messages 
appeared  at  approximately  20,  26,  and  32  minutes  into  the  scenario.  Participants  were 
specifically  tasked  to  monitor  the  Email/Intel  traffic. 


Simulator  and  Scenario 

The  Distributed  Dynamic  Decision-making  (DDD)  environment  was  used  to  simulate 
two  mission  scenarios.  The  DDD  is  a  distributed  client/server  simulator  that  allows 
extensive  mission  customization  in  order  to  investigate  individual  and  team  performance 
in  an  operationally  rich,  experimentally  valid  environment.  In  general,  DDD  simulations 
involve  individual  and  team  decision-making  about  complex  situations  based  on 
information  and  resources  provided  by  both  the  simulation  and  other  team  members 
(Serfaty  &  Kleinman,  1985;  Kleinman  &  Serfaty,  1989).  The  simulation  enables  the 
manipulation  of  variables  such  as  organizational  structure  and  mission  scenario  tasking. 
A  variety  of  performance  measures  can  be  recorded  within  the  DDD  (i.e.,  tasks 
processed,  latencies,  and  accuracies)  to  assess  performance.  The  two  scenarios  we  used 
involve  land,  sea,  and  air  operations  to  prepare  the  battle  space  for  insertion  of  forces  for 
follow-on  actions. 


Dependent  Measures 

The  DDD  simulator  enables  the  measurement  of  several  variables  related  to  individual  and 
team  performance  such  as  latency  to  process  a  task,  accuracy  in  processing  a  task, 
percentage  of  tasks  process,  and  percentage  of  task  processed  at  100%  accuracy.  In  this 
effort,  we  focused  on  the  percent  of  tasks  processed  (the  number  of  tasks  attacked/the 
number  of  tasks  that  arrived  in  the  scenario). 


To  assess  whether  participants  were  sensitive  to  the  embedded  confirming  and  disconfirm 
information,  employed  a  contrast  and  adjustment  heuristic  when  dealing  with  confirming 
and  disconfirming  messages,  and  experience  an  order  effect,  we  assessed  participants’ 
beliefs  about  the  likelihood  of  an  enemy  counter-attack  throughout  the  scenario. 

Participants  responded  to  four  pop-up  windows  soliciting  their  beliefs.  The  response  scale 
in  the  pop-up  window  started  at  0%  likelihood  of  an  attack  and  increased  in  10% 
increments  to  100%  likelihood  of  an  attack.  The  pop-up  windows  were  timed  to  appear  at 
500,  1000,  1500,  and  2000  seconds  into  the  scenario.  However,  only  the  second  and  last 
estimates  were  used  in  analysis.  The  second  estimate,  17  minutes  or  about  halfway  through 
the  scenario,  assessed  the  impact  of  the  first  three  messages,  where  as  the  last  estimate  at 
34  minutes  or  about  one  minute  form  the  end  of  the  scenario,  captured  the  impact  of  the 
last  three  messages. 


Procedure 


Participants  received  DDD  “buttonology”  training  followed  by  training  designed  to 
provide  the  skills  necessary  to  perform  a  functional  scenario.  Teams  next  engaged  in  a 
75  minute  data  collection  session,  and  in  a  counter-balanced  order  performed  two  similar 
scenarios:  one  under  high  information  load  and  one  under  low  information  load. 

Teams  were  told  that  their  primary  mission  was  to  engage  in  information  gathering  to 
achieve  and  maintain  good  situational  awareness  regarding  enemy  activities  in  order  to 
discern  if  the  enemy  intended  to  launch  an  attack  against  follow-on  Blue  forces.  To 
accomplish  this,  team  members  were  required  to  monitor  the  Email/Intel  traffic  and  fuse 
the  information  from  the  various  messages  to  maintain  situational  awareness.  Every  500 
seconds,  each  team  member  responded  to  a  request  from  the  Commander  of  the  Joint 
Task  Force  requesting  the  likelihood  that  the  enemy  would  launch  an  attack.  Teams  were 
also  instructed  to  protect  their  assets  against  enemy  attack  and  to  prevent  the  enemy  from 
attacking  protected  zones.  As  a  secondary  task  the  teams  were  asked  to  complete  the 
mission  tasks  which  included  destroying  the  enemy’s:  command  center,  air  base,  bridge, 
naval  base,  and  seaport,  aw  well  as,  finding  and  destroying  SCUD  missile  launchers. 
Participants  were  told  that  it  was  likely  that  they  would  not  complete  all  these  tasks. 


Results 

Results  from  the  organizational  structure  X  information  load  X  trial  analysis  for  the 
percentage  of  tasks  processed  are  depicted  in  Fig.  1.  Unexpectedly  the  ANOVA  showed 
a  significant  trial  effect  (p  <  .005)  indicating  that  teams  performed  about  25%  better  in 
trial  2  than  trial  1 .  Apparently  teams  were  still  learning  how  to  perform  the  mission  tasks 
during  trial  1 .  The  results  further  show  that,  contrary  to  prediction,  the  organizational 
structure  with  an  ISR  coordinator  did  not  out  perform  the  traditional  CWC-like  structure 
without  an  ISR  coordinator.  Examining  the  interaction  (p  <  .07)  within  trial  1  we  see  that 
the  teams  without  an  ISR  coordinator  performed  at  a  higher  level  than  the  teams  with  an 
ISR  coordinator,  and  did  so  more  when  information  load  was  high  than  low.  In  trial  2  the 
pattern  is  quite  different;  teams  in  both  organizational  structures  performed  at  the  same 
level  when  information  load  was  low  and  almost  at  the  same  level  when  information  load 
was  high.  Moreover,  the  increase  in  performance  between  trials  1  and  2  for  teams 
without  the  ISR  is  small,  while  the  same  difference  for  teams  with  the  ISR  coordinator  is 
much  larger.  The  steep  increase  in  performance  for  teams  under  the  structure  with  an 
ISR  coordinator  makes  us  wonder  if  there  had  been  a  third  trial,  whether  their 
performance  would  have  surpassed  that  of  teams  under  the  structure  without  an  ISR 
coordinator.  Perhaps  the  team  members  comprising  the  structures  with  an  ISR 
coordinator  were  still  learning  the  aspects  of  their  novel  structure? 


Figure  1.  Percentage  of  Mission  Tasks  Attacked  as  a  Function  of  Organizational 
Structure  and  Information  Load  for  Trials  1  and  2 

A  trial  X  information  load  X  order  X  estimate  time  MANOVA  was  conducted  to  evaluate 
the  effects  of  information  load  on  tactical  judgments  and  whether  participants  would  be 
prone  to  the  order  effects  described  by  Hogath  and  Einhom’s  (1992).  The  analysis 
revealed  several  significant  main  effects  and  interactions.  Of  particular  interest  to  our 
predictions  was  the  significant  (p  <  .06)  order  X  estimate  time  interaction  plotted  in 
Figure  2.  As  predicted,  the  confirming-disconfirming  and  disconfirming-  confirming 
confirmation  orders  elicited  different  strengths  of  belief  for  an  attack  from  the 
participants,  even  though  the  number  of  confirming  and  disconfirming  pieces  of  evidence 
was  the  same.  Further  investigation,  summarized  in  Figs  3  and  4,  revealed  that  high 
information  load  appears  to  heighten  the  order  effect.  Figure  3  shows  the  plot  for  the 
order  X  estimate  time  contrast  when  information  load  is  low.  There  is  no  evidence  of  an 
order  effect.  The  pattern  in  Fig.  4  shows  that  participants  experiencing  the 
disconfirming-  confirming  confirmation  order  held  a  final  judgment  indicating  a 
moderately  strong  probability  (i.e.,  0.75)  of  an  enemy  counter-attack.  However, 


Figure  2.  Likelihood  of  Attack  Probability  for  Confirmation  Order  (Confirming- 
Disconfirming  and  Discontinuing-  Confirming)  and  Estimate  Times 

participants  who  experienced  the  confirming-disconfirming  confirmation  order  arrived  at 
a  final  judgment  that  indicated  they  were  undecided  (i.e.,  probability  of  attack  =  0.50)  if 
an  enemy  counter-attack  would  occur.  Recall  that  the  confirmation  information  had  to  be 
gleaned  from  the  Email/Intel  traffic  over  75  messages.  Even  though  the  two  groups  of 
participants  received  the  same  information  only  in  a  different  order,  the  ordering  of 
confirming  followed  by  discontinuing  and  the  ordering  of  disconfirming  followed  by 
confirming  information  led  the  respective  groups  to  arrive  at  different  positions 
concerning  the  likelihood  that  the  enemy  would  launch  an  attack,  supporting  predictions 
from  Hogath  and  Einhom’s  (1992).  contrast-inertia  heuristic  model. 
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Figure  3.  Likelihood  of  Attack  Probability  for  Confirmation  Order  (Confirming- 
Disconfirming  and  Disconfirming-  Confirming)  and  Estimate  Times  When  Information 

Load  is  Low 


Figure  4.  Likelihood  of  Attack  Probability  for  Confirmation  Order  (Confirming- 
Disconfirming  and  Discontinuing-  Confirming)  and  Estimate  Times  When  Information 

Load  is  High 

Discussion 

We  hypothesized  that  an  organizational  structure  that  includes  an  ISR  coordinator  would 
perform  at  a  higher  level  than  an  organizational  structure  that  does  not  include  an  ISR 
coordinator.  Performance  results  from  the  current  experimental  effort  did  not  support 
this  hypothesis.  However,  the  improvement  of  the  structure  with  an  ISR  coordinator 
from  trial  1  to  trial  2  was  steeper  than  that  of  the  traditional  organization  and  by  trial  2 
caught  up  to  the  performance  of  the  structure  without  an  ISR  coordinator  in  both  low  and 
high  information  load  conditions.  This  may  indicate  that  the  presence  of  an  ISR 
coordinator  does  facilitate  performance  improvement  and  with  more  experience  and 
practice  teams  that  have  such  an  ISR  coordinator  would  surpass  the  performance  of  the 
teams  without  the  ISR  coordinator. 

We  suggest  that  a  reason  for  the  initial  weak  performance  of  the  organization  with  an  ISR 
coordinator  was  the  lack  of  familiarity  with  the  position  of  an  ISR  coordinator. 
Organizations  with  ISR  coordinators  are  new  and  our  Navy  officers  did  not  have 
experience  with  such  an  organization.  The  CWC  doctrine,  however,  is  currently  a  key 
tenet  of  Navy  training,  and  the  officers  fulfilling  the  CWC  roles  would  certainly  be 
familiar  with  these  positions.  Entin,  Serfaty,  and  Kerrigan  (1998)  describe  results  similar 
to  this.  Teams  comprised  of  military  officers  performed  at  a  higher  level  in  traditional  C2 
organizational  structures  compared  to  non-traditional  C2  structures  even  though  the  non- 
traditional  structures  had  been  optimized  to  match  the  mission  because  the  officers  were 
familiar  and  practiced  with  the  traditional  structures. 

Our  investigations  into  information  load  and  the  sequential  revision  of  judgments  showed 
that  participants  exhibited  the  order  effect  heuristic  even  though  the  confirmatory  and 
disconfirmatory  evidence  was  embedded  in  Email/Intel  traffic  and  not  specifically  called 
out.  We  are  not  aware  of  any  findings  demonstrating  an  order  effect  when  the  salient 
evidence  is  embedded  in  flood  of  information.  It  also  appears  that  the  order  effect  is 


strengthen  by  high  information  load  and  does  not  occur  under  low  information  load.  One 
speculation  is  that  the  higher  stress  brought  on  by  high  information  load  functions  to 
increase  individuals’  susceptibility  to  heuristic  errors  like  the  order  effect.  These  results 
suggest  a  problem  related  to  network  centric  warfare’s  tendency  to  produce  high 
information  loads  that  must  be  addressed. 
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Hum  m  ■  Cirtitrid  EngifPitnng 

FORCEnet  Principles  Hold  Promise 

■  Modeling  &  simulation  effort 
conducted  for  SSG-XXI, 
predict: 

■  An  increase  in  shared  knowledge 
in  a  FORCEnet  structure 

■  Allow  organizations  to  respond 
more  effectively  to  new  & 
unexpected  situations 

'  More  adaptable  structures 

'  Will  handle  complex  tasks  that  require  coordination  more 

effectively  than  current  organization  structures 


I  n  FORCEnet  collaboration  as  a  way  of  life 


FORCEnet  Drives  Investigation  of  New 

C2  Structures 
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>  Modeling  examined  several  structural 
changes 

■  One  example  -  creations  of  an  intelligence, 
surveillance,  and  reconnaissance  (ISR) 
coordinator 

»  Could  increase  mission  performance  by  as 
much  as  25% 

■  A  primary  goal  of  this  research  was  to 
empirically  test  this  model  prediction 

■  ISR  vs.  no  ISR  organizational  structures 


Network  Centric  Warfare  Spawns  High 

Information  Loads 
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1  Increased  volume  of  information  that 
commanders  must  deal  with 

•  High  information  load  can  denigrate  situation 
assessment  &  decrease  mission  performance 

■  Second  study  goal:  investigate  effects  of  high 
information  load  on  decision  making  &  mission 
performance 

■  Manipulate  information  load  across  the  two 
organizational  structures  (ISR  &  no  ISR) 

■  Embedded  tactical  judgment  task  to  focus  on  critical 
aspects  of  decision  making  under  different  information 
loads 
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^  Huftin  •  C-irtitrid  Engifllitring 

Tactical  Judgment  Task 


■  Addresses  the  process  of  sequential  revision  of  belief 

■  Task:  estimate  probability  that  the  enemy  would  launch  a 
counter-attack 

*  Information  necessary  for  the  judgment  task  embedded  in 
information  flow 

■  Two  orders  of  information:  3  confirming  followed  by  3  disconfirming 
&  3  disconfirming  followed  by  3  confirming  messages 

■  Likelihood  of  attack  assessed  at  middle  &  end  of  scenario 

■  A  contrast-inertia  model  (Hogath  &  Einhorn,  1992) 
postulated  to  describe  sequential  revision  process 

■  The  “order  effect”  -  a  heuristic  error 

•  Sequential  order  of  confirming  or  disconfirming  evidence  can 
have  a  profound  effect  on  participants’  judgments 

■  Not  sure  how  information  load  will  interact  with  order 
effect  -  two  speculations 


Experimental  Design 
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Independent  variables: 

■  Organizational  Structures  (both  functional) 

■  ISR  Coordinator  vs.  No  ISR  coordinator  (Surface  Warfare 
Commander) 

■  Information  Load 

■  High  (9  messages  per  minute) 

-  Low  (3.5  messages  per  minute) 

-  Counter-balanced  across  trials 

■  Confirmation  order 

■  Confirm-disconfirm 

■  Disconfirm-confirm 
Between  Ss  -  Nested 

Primary  performance 
measure 

"  Percentage  of  task 
attacked  with  100% 
accuracy 
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Within-Subjects 

4  teams 
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Simulation  Environment 


DDD  simulation:  modified  A2C2  Exp  8  Scenarios 


Scenarios  used  involved 
land,  sea,  &  air  operations 
to  prepare  battle  space 
for  insertion  of  follow  on 
forces 

Primary  mission  was  to 
engage  in  information 
gathering,  achieve  & 
maintain  good  SA,  discern 
“  if  enemy  planned  launch  a 
counter-attack  -  monitor 
Email/Intel  traffic 

Secondary  task:  complete 


1 

mission  tasks 

a 

£ 

Presence  of  an  ISR  Coordinator: 

Performance  Results 
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Contrary  to  expectations  the  traditional  org.  structure  with  no  I  SR 
coordinator  out  performed  the  org.  structure  with  an  ISR 
coordinator  -  most  evident  in  trial  1  (p  <  .055) 
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Trial  2 


In  trial  2,  pattern  quite  different:  teams  in  both  organizational  structures  performed 
at  about  the  same  level  in  the  high  &  low  information  load  conditions 

"  Steep  improvement  (38%  f  p  <  .05)  in  performance  for  org.  with  I  SR  coord,  in  high 
information  load  condition 

Speculate  that  with  sufficient  training  &  practice  org.  with  I  SR  coord,  mighl 
prove  superior 


Tactical  Judgment 
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As  predicted  the  confirming-disconfirming  &  disconfirming 
confirmation  orders  elicited  different  strengths  of  belief  for  an 
enemy  attack  ( p  <  .  06) 
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Tactical  Judgment  and  Information  Load 

"  No  evidence  of  an  order  effect  when  information  load  is  low 
*  Significant  contrast  (order  effect)  when  information  load  is  high 


Low  I  nformation  Load 


High  Information  Load 


The  results  demonstrated  an  order  effect  despite  the  fact: 
Participants  had  to  glean  the  information  from  the  Email/Intel 
traffic  of  over  75  messages 

■ 

Participants  received  the  same  information  only  in  a 
different  order 


Conclusions 
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•  Performance  results  did  not  support  the  prediction  that 
an  org.  with  an  ISR  coord,  would  out  perform  an  org. 
without  an  ISR  coord. 

•  However,  org.  with  an  ISR  coord.  Caught  up  to  the  org.  without 
an  ISR  coord,  in  trial  2  in  both  low  &  high  information  load 
conditions 

■  Steep  improvement  may  indicate  presence  of  an  ISR  coord, 
does  facilitate  performance  once  sufficient  training  &  practice 

have  occurred 

■  Participants  exhibited  an  order  effect  (a  heuristic  error) 
even  though  the  confirmatory  and  disconfirmatory 
evidence  was  embedded  in  Email/Intel  traffic 

*  Order  effect  heuristic  error  strengthen  by  high  information  load 

■  Suggest  a  problem  related  to  network  centric  warfare  that  will 
have  to  be  address 


